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We present the results of a search for the production of an excited state of the muon, /i*, in proton 
antiproton collisions at '/s = 1.96 TeV. The data have been collected with the DO experiment at the 
Fermilab Tevatron Collider and correspond to an integrated luminosity of approximately 380 pb~^. 
We search for /i* in the process pp — > /J*/^, with the p* subsequently decaying to a muon plus 
photon. No excess above the standard model expectation is observed in data. Interpreting our data 
in the context of a model that describes /i* production by four-fermion contact interactions and 
^* decay via electroweak processes, we exclude production cross sections higher than 0.057 pb - 
0.112 pb at the 95% confidence level, depending on the mass of the excited muon. Choosing the 



scale for contact interactions to be A = 1 TeV, excited muon masses below 618 GeV are excluded. 
PACS numbers: 12.60.Rc, 14.60.Hi, 12.60.-i, 13.85.Rm 



An open question in particle physics is the observed 
mass hierarchy of the quark and lepton SU(2) doublets 
in the standard model (SM). A commonly proposed ex- 
planation for the three generations is a compositeness 
model 1] of the known Icptons and quarks. According 
to this approach, a quark or lepton is a bound state of 
three fermions, or of a fermion and a boson Q- Due to 
the underlying substructure, compositeness models im- 
ply a large spectrum of excited states. The coupling of 
excited fermions to ordinary quarks and leptons, result- 
ing from novel strong interactions, can be described by 
contact interactions (CI) with the effective four-fermion 
Lagrangian ^ 
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where j^ is the fermion current 



+ h.c. + {L-f R). 

The SM and excited fermions are denoted by / and /*, 
respectively; g^ is chosen to be 47r, the rj factors for the 
left-handed currents are conventionally set to one, and 
the right-handed currents are set to zero. The compos- 
iteness scale is A. 

Gauge mediated transitions between ordinary and ex- 
cited fermions can be described by the effective La- 
grangian jij 
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where G^j,, W^^i and B^y are the field strength tensors of 
the gluon, the SU(2) and U(l) gauge fields, respectively; 
/s, / and /' are parameters of order one. 

The present analysis considers single production of an 
excited muon /i* in association with a muon via four- 
fermion CI, with the subsequent electroweak decay of 
the /i* into a muon and a photon (Fig. ^. This de- 
cay mode leads to the fully reconstructable and almost 
background- free final state /i/J.7. With the data consid- 
ered herein, collected with the DO detector at the Fermi- 
lab Tevatron Collider in pp collisions at \/s = 1.96 TeV, 
the largest expected SM background is from the Drell- 
Yan (DY) process pp -^ Zf-f* — > /i+/i^(7), with the final 
state photon radiated by either a parton in the initial 
state p or p, or from one of the final state muons. This 
background can be strongly suppressed by the applica- 
tion of suitable selection criteria. Other backgrounds are 
small. 

Excited muons have been searched for unsuccessfully 
previously |[J|, e.g. at the LEP e+e~ collider; however the 



reach has been limited by the center-of-mass energy avail- 
able to m^. < 190 GeV. Searches for quark-lepton com- 
positeness via deviations from the Drell-Yan cross section 
have excluded values of A of up to « 6 TeV depending on 
the chirality [5j. The present analysis is complementary 
to those results in the sense that an exclusive channel 
and different couplings (77 factors) are probed. The CDF 
collaboration has recently presented results [g for the 
production of excited electrons which will be discussed 
later. 
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FIG. 1: Four-fermion contact interaction qq -^ fJ^'fJ^, and elec- 
troweak decay /i* -^ n'y. On the right, the relative contribu- 
tion of decays via CI and via electroweak interactions (EW) 
as a function of m^» /A is shown. 

For the simulation of the signal a customized ver- 
sion of the PYTHIA event generator Q is used, following 
the model of Q- The branching fraction for the decay 
/i* — > ^7 normalized to all gauge particle decay modes is 
30% for masses above 300 GeV, and for smaller fi* masses 
it increases up to 73% at m^* = 100 GeV. Decays via con- 
tact interactions, not implemented in pythia, contribute 
between a few percent of all decays for A ^ to^. and 
92% for A = m^. [1I3 (see Fig.^. This has been taken 
into account for the signal expectation. The leading or- 
der cross section calculated with pythia has been cor- 
rected to next-to- next-to-leading order (NNLO) |g. [lO|: 
the corresponding correction factor varies between 1.430 
(1.468) for m^. = 100 GeV (200 GeV) and 1.312 for 
TOp. = 1 TeV. The total width is greater than 1 GeV for 
100 GeV < rrif,. < 1000 GeV, thus hfetime effects can be 
neglected. For the values of nif^* and A studied here, the 
total width is always less than 10% of m^. 3|. 

The dominant SM background process at all stages of 
the selection is DY production of ji^ n" pairs. This back- 
ground, as well as diboson [WW, WZ, ZZ) production, 
has been simulated with the pythia Monte Carlo (MC) 
program. The DY expectation has been corrected us- 
ing the NNLO calculation from ^ . For diboson produc- 
tion, the next-to-leading order cross sections from [IJ 
are used. Monte Carlo events, both for SM and signal, 
have been passed through a detector simulation based 
on the GEANT 12] package, and reconstructed using the 
same reconstruction program as the data. The CTEQ5L 
parton distribution functions (PDF) [l^ are used for the 
generation of all MC samples. 

The analysis is based on the data collected with the DO 
detector |lj| between August 2002 and September 2004, 



corresponding to an integrated luminosity of 380 pb~^. 
The DO detector includes a central tracking system, com- 
prised of a silicon microstrip tracker (SMT) and a central 
fiber tracker (CFT), both located within a 2 T supercon- 
ducting solcnoidal magnet. The SMT has « 800,000 
individual strips, with typical pitch of 50 — 80 /im, and 
a design optimized for tracking and vertexing capability 
at pseudorapidities ,15| of |r/| < 2.5. The CFT has eight 
coaxial barrels, each supporting two doublets of scintil- 
lating fibers of 0.835 mm diameter, one doublet being 
parallel to the collision axis, and the other alternating by 
±3° relative to the axis. Three liquid argon and uranium 
calorimeters provide coverage out to \ri\ « 4.2: a central 
section covering \r]\ up to « 1.1, and two end calorime- 
ters. A muon system resides beyond the calorimetry, and 
consists of a layer of tracking detectors and scintillation 
trigger counters before 1.8 T iron toroids, followed by two 
similar layers after the toroids. Tracking at \ri\ < 1 relies 
on 10 cm wide drift tubes, while 1 cm mini-drift tubes 
are used at 1 < I77I < 2. Luminosity is measured using 
scintillator arrays located in front of the end calorimeter 
cryostats, covering 2.7 < \i]\ < 4.4. 

Trigger and data acquisition systems are designed to 
accommodate the high luminosities of the Tevatron Run 
II. Based on information from tracking, calorimetry, and 
muon systems, the output of the first two levels of the 
trigger is used to limit the rate for accepted events to 
< 1 kHz, relying on hardware and firmware. The third 
and final level of the trigger uses software algorithms and 
a computing farm to reduce the output rate to ~ 50 Hz, 
which is written to tape. 

Efficiencies for muon and photon identification and 
track reconstruction are determined from the simula- 
tion. To verify the simulation and to estimate systematic 
uncertainties, the efficiencies have also been calculated 
from data samples, using Z —^ fJ-'^ n~ candidate events 
and inclusive dimuon events for muons and tracks, and 
Z —^ e^e~ events to determine the efficiency of recon- 
structing electrons. We assume that the different re- 
sponse for electrons and photons in the calorimeter is 
properly modelled by the simulation. The transverse 
(with respect to the beam axis) momentum resolution 
of the central tracker and the energy resolution of the 
calorimeter have been tuned in the simulation to repro- 
duce the resolutions observed in the data using Z ^ ££ 
{£ = e, n) events. 

The process pp —^ /i*^ with n* -^ ^7 leads to a final 
state with two highly energetic isolated muons and a pho- 
ton. We require two muons to be identified in the muon 
system and each matched to a track in the central track- 
ing system with transverse momentum p^ > 15 GeV. 
The events have been collected with Level 1 trigger con- 
ditions requiring two muons detected by the muon scin- 
tillation counters, with at least one muon with tightened 
criteria identified by the Level 2 trigger, and requiring a 
segment reconstructed in the muon system above certain 
Pt thresholds and/or a track in the central tracking sys- 
tem above certain px thresholds at Level 3. The trigger 
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FIG. 2: a) Invariant dimuon mass distribution in tlie dimuon 
data sample compared to the SM expectation, b) invariant 
mass of the leading muon and the photon in the /i^7 sam- 
ple, for data (points with statistical uncertainties), SM back- 
grounds (DY and diboson production, shaded histograms, as 
well as the uncertainty due to jets misidentified as photons), 
and the expected signal for m^* = 400 GeV and A = 1 TeV. 



efficiency has been determined from independent data 
samples for each trigger object (muon) and trigger level 
separately. The overall trigger efficiency which is applied 
to the simulation is found to be 88 ± 6% for the signal 
after application of all selection criteria. 

Timing information from the muon scintillation coun- 
ters is used in order to reject cosmic ray background. 
Since the signal is expected to produce isolated muons, 
at least one of the muons is required to be isolated: 
the amount of energy deposited in the calorimeter along 
the muon direction in a hollow cone with inner radius 
A7^ = 0.1 (A7^ = ^(A?7)2 + (A(^)2) and outer radius 
ATZ — 0.4 is required to be less than 2.5 GeV, and 
the sum of the transverse momenta of tracks within a 
cone of ATZ — 0.5 has to be below 2.5 GeV, excluding 
the muon track. The cumulative efficiency of the muon 
and track reconstruction and muon identification is found 
to be 88 ± 4% per muon, and the isolation condition is 
95 ± 4% efficient. The selected dimuon sample contains 
24853 events, whereas 23200 ± 2700 events are expected 
from DY processes, and 34 ± 4 events are expected from 
diboson production. The invariant dimuon mass distri- 
bution is shown in Fig. El a). 

Next, a photon is identified in the event as an iso- 
lated cluster of calorimeter energy with a characteristic 
shower shape and at least 90% of the energy deposited in 
the electromagnetic section of the calorimeter. The isola- 
tion condition is (Etot(0.4) -£;om(0.2))/£;em(0.2) < 0.15, 
where i?tot(0.4) and £"0111(0.2) denote the energy de- 
posited in the calorimeter and only its electromagnetic 
section in cones of size ATZ = 0.4 and 0.2, respectively. 
The transverse energy Et must be larger than 16 GeV, 
no track is allowed to be matched to the photon candi- 
date with a x^ probabfility of greater than 0.1%, and the 
sum of the transverse momenta of tracks within a hollow 
cone defined by 0.05 < AT?. < 0.4 around the photon di- 
rection has to be below 2 GeV to further ensure isolation. 
The photon candidate is required to be separated from 




FIG. 3: For the /1/17 sample, a) the distribution of the leading 
muon pt, and b) the photon Et- Shown are the data as points 
with statistical uncertainties, the dominant SM background 
(DY, shaded histogram, also shown is the uncertainty due 
to jets misidentified as photons), and the expected signal for 
m„. = 400 GeV and A = 1 TeV. 
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1000 


440 





0.003 ±0.001 


14.4 ±1.8 



TABLE I; For different values of m^* , the final selection re- 
quirement on the invariant mass of the leading muon and the 
photon, the remaining data events, the SM expectation, and 
the signal efficiency. The quoted uncertainties include statis- 
tical and systematic uncertainties added in quadrature. 



the muon candidates in the event by at least ATZ = 0.4, 
and has to be reconstructed within the central part of 
the calorimeter (jryj < 1.1). 

After this selection, we expect 65 ± 8 events from DY 
processes, and less than one event from diboson produc- 
tion. To estimate the possible additional background 
from jets misidentified as photons and not included in 
the simulation, the misidentification rate has been deter- 
mined from an inclusive jet data sample; this rate applied 
to the dimuon plus jet sample results in 39±5 such events 
in the /i/i7 selection. As a function of Et, the photon fake 
rate is about 0.5% per jet at low Et, and is negligible 
above « 80 GeV. The background from jets misidentified 
as photons is treated as a systematic uncertainty, result- 
ing in a total SM expectation of 65 ± 8 ^q^ events. We 
find 90 events in the data, in good agreement with the 
expectation. The invariant mass of the leading muon and 
the photon is shown in Fig. [21b) for the data, SM expec- 
tation, and signal expectation for ttt.^. = 400 GeV and 
A = 1 TeV. The pt distribution of the leading muon and 
the Et distribution of the photon are shown in Fig. [3| 

Additional selection criteria are applied to reduce the 
remaining SM background. The photon Et is required to 
be larger than 27 GeV. The efficiency to identify a pho- 
ton is constant at about 90% above this value. The final 
discriminant to suppress remaining SM backgrounds is 
the invariant mass of the leading muon and the photon. 
For masses m^. above « 300 GeV, the leading muon is 
predominantly the muon from the /i* decay. In order to 
maximize the sensitivity of the analysis, the signal ex- 
pectation is calculated for A = 1 TeV, the background 
including DY processes and diboson production is con- 
sidered, and a cut value is chosen for each value of m^* . 
The result is shown in Table Q] along with the SM ex- 
pectation for the number of data events and the signal 
efficiency, which varies between 8% and 15%. 

The dominant systematic uncertainties are as follows. 
The uncertainty on the SM cross sections is dominated 
by the DY process and the uncertainty from the choice of 



PDF and renormalization and factorization scales (4%). 
Muon reconstruction and identification have an uncer- 
tainty of 4% per muon, and a 3% error is assigned to the 
photon identification. The uncertainty due to the trig- 
ger efficiency is 7%. The integrated luminosity is known 
to a precision of 6.5% [l^- The uncertainty due to jets 
misidentified as photons is dominant after all selection 
criteria for to^» up to 400 GeV: for m^* = 100 GeV 
(400 GeV), 0.097 (0.008) such "fake" photons are ex- 



pected, while for m^ 



500 GeV and above this back- 



ground is negligible (< 10~ events). The uncertainty 
on the signal cross section is estimated to be 10%, con- 
sisting of PDF uncertainties and unknown higher order 
corrections. 

Since no events are found in the data, in agreement 
with the SM expectation, we set 95% confidence level lim- 
its on the fi* production cross section times the branching 
fraction into fij. A Bayesian technique [I3I is used, tak- 
ing into account all uncertainties and treating them as 
symmetric for simplicity. The resulting limit as a func- 
tion of TTip* is shown in Fig. ^together with predictions 
of the contact interaction model for different choices of 
the scale A. For A = 1 TeV (A = m^. ), masses below 
618 GeV (688 GeV) are excluded. In Fig.|31the excluded 
region in terms of A and m^* is shown. 

The CDF collaboration has recently searched [3] for 
the production of excited electrons, and obtained com- 
parable cross section limits, but the CDF mass limit of 
TOg. > 879 GeV at 95% C.L. for A = mg. cannot be 
directly compared to ours for two reasons. The cross 
section calculated with the version of pythia used by 
CDF is a factor of two higher than in subsequent ver- 
sions corrected by the pythia authors. Furthermore, 
CDF assumes that decays via contact interactions can 
be neglected, while in our analysis such decays are taken 
into account in the calculation of the branching frac- 
tion ^* — > /17, following 0, I3. If we adjusted our re- 
sult for these two differences, we would obtain a limit of 



m^, > 890 GeV at 95% C.L. for A 



(-M*- 



DQ 380 pb 



m„.>618GeV 




FIG. 4: The measured cross section x branching fraction 
limit, compared to the contact interaction model prediction 
for different choices of A. For the case A = 1 TeV, the theo- 
retical uncertainty of the model prediction is indicated. 



In summary, we have searched for the production of 
excited muons in the process pp —^ fx* fi with /x* -^ /X7, 
using 380 pb^^ of data collected with the DO detector. 
We find no events in the data, compatible with the SM 
expectation, and set limits on the production cross sec- 
tion times branching fraction as a function of the mass 
of the excited muon. For a scale parameter A = 1 TeV, 
masses below 618 GeV are excluded, representing the 
most stringent limit to date. 
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FIG. 5: The region in the plane of A and nip* excluded by 
the present analysis. 
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